A representer theorem for deep kernel learning

In this paper we provide a representer theorem for a concatenation of (linear combinations of) kernel functions of reproducing kernel Hilbert spaces. This fundamental result serves as a first mathematical foundation for the analysis of machine learning algorithms based on compositions of functions. As a direct consequence of this new representer theorem, the corresponding infinite-dimensional minimization problems can be recast into (nonlinear) finite-dimensional minimization problems, which can be tackled with nonlinear optimization algorithms. In this context, we show how concatenated machine learning problems can be reformulated as neural networks and how our representer theorem applies to a broad class of state-of-the-art deep learning methods. We use our results to establish deep learning algorithms based on the composition of functions from reproducing kernel Hilbert spaces.